{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qnyTxjK_GbOD"
   },
   "source": [
    "# Beyond Hello World, A Computer Vision Example\n",
    "# 超越Hello World，一个计算机视觉的实例\n",
    "In the previous exercise you saw how to create a neural network that figured out the problem you were trying to solve. This gave an explicit example of learned behavior. Of course, in that instance, it was a bit of overkill because it would have been easier to write the function Y=2x-1 directly, instead of bothering with using Machine Learning to learn the relationship between X and Y for a fixed set of values, and extending that for all values.\n",
    "\n",
    "But what about a scenario where writing rules like that is much more difficult -- for example a computer vision problem? Let's take a look at a scenario where we can recognize different items of clothing, trained from a dataset containing 10 different types.\n",
    "\n",
    "在上一个练习中，你看到了如何创建一个神经网络，让它学会图解决某个问题。那是一个计算机习得行为的明确例子。当然，就那个例子而言，运用神经元网络来解决有点大炮轰苍蝇的意思，因为直接写出函数Y=2x-1会更容易，而不是费力地使用机器学习来学习X和Y之间对于一组固定值的关系，并将其扩展到所有值。\n",
    "\n",
    "但是，在一个场景下，写出明确的函数规则要困难得多，比如计算机视觉问题。让我们来看这样一个场景，让计算机识别不同的服装用品（有提包、鞋子、裤子等10类物品）。我们将用包含10种不同类型的物品图片的数据集来训练一个神经元网络，实现分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "H41FYgtlHPjW"
   },
   "source": [
    "## Start Coding\n",
    "## 开始编程\n",
    "Let's start with our import of TensorFlow\n",
    "\n",
    "首先导入TensorFlow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1577,
     "status": "ok",
     "timestamp": 1550587660097,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "q3KzJyjv3rnA",
    "outputId": "7a1ceed7-66f6-4a43-b3f4-26d54bd95817"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n_n1U5do3u_F"
   },
   "source": [
    "The Fashion MNIST data is available directly in the tf.keras datasets API. You load it like this:\n",
    "通过tf.keras的API可以直接获得Fashion MNIST数据集，像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PmxkHFpt31bM"
   },
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.fashion_mnist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GuoLQQBT4E-_"
   },
   "source": [
    "Calling load_data on this object will give you two sets of two lists, these will be the training and testing values for the graphics that contain the clothing items and their labels.\n",
    "在mnist对象上调用load_data方法会得到两个元组，各自包含两个列表。这些列表存储了服装用品的训练与测试图像数据及标签值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1381,
     "status": "ok",
     "timestamp": 1550587667916,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "BTdRgExe4TRB",
    "outputId": "104aab14-aa09-4ed2-806c-38807fbac030"
   },
   "outputs": [],
   "source": [
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rw395ROx4f5Q"
   },
   "source": [
    "What does these values look like? Let's print a training image, and a training label to see...Experiment with different indices in the array. For example, also take a look at index 42...that's a a different boot than the one at index 0\n",
    "这些值是什么样子的呢？让我们打印一个训练图像，和一个训练标签来看看......用数组中不同的索引做实验。例如，也可以看看索引42......那也是另一张鞋子的图片。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1238
    },
    "colab_type": "code",
    "collapsed": true,
    "executionInfo": {
     "elapsed": 357,
     "status": "ok",
     "timestamp": 1550587722332,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "FPc9d3gJ3jWF",
    "jupyter": {
     "outputs_hidden": true
    },
    "outputId": "5efae963-162d-4418-c001-2e8d6af83f40"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "[[0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.00392157 0.         0.         0.05098039 0.28627451 0.\n",
      "  0.         0.00392157 0.01568627 0.         0.         0.\n",
      "  0.         0.00392157 0.00392157 0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.01176471 0.         0.14117647 0.53333333 0.49803922 0.24313725\n",
      "  0.21176471 0.         0.         0.         0.00392157 0.01176471\n",
      "  0.01568627 0.         0.         0.01176471]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.02352941 0.         0.4        0.8        0.69019608 0.5254902\n",
      "  0.56470588 0.48235294 0.09019608 0.         0.         0.\n",
      "  0.         0.04705882 0.03921569 0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.60784314 0.9254902  0.81176471 0.69803922\n",
      "  0.41960784 0.61176471 0.63137255 0.42745098 0.25098039 0.09019608\n",
      "  0.30196078 0.50980392 0.28235294 0.05882353]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.00392157\n",
      "  0.         0.27058824 0.81176471 0.8745098  0.85490196 0.84705882\n",
      "  0.84705882 0.63921569 0.49803922 0.4745098  0.47843137 0.57254902\n",
      "  0.55294118 0.34509804 0.6745098  0.25882353]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.00392157 0.00392157 0.00392157\n",
      "  0.         0.78431373 0.90980392 0.90980392 0.91372549 0.89803922\n",
      "  0.8745098  0.8745098  0.84313725 0.83529412 0.64313725 0.49803922\n",
      "  0.48235294 0.76862745 0.89803922 0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.71764706 0.88235294 0.84705882 0.8745098  0.89411765\n",
      "  0.92156863 0.89019608 0.87843137 0.87058824 0.87843137 0.86666667\n",
      "  0.8745098  0.96078431 0.67843137 0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.75686275 0.89411765 0.85490196 0.83529412 0.77647059\n",
      "  0.70588235 0.83137255 0.82352941 0.82745098 0.83529412 0.8745098\n",
      "  0.8627451  0.95294118 0.79215686 0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.00392157 0.01176471 0.\n",
      "  0.04705882 0.85882353 0.8627451  0.83137255 0.85490196 0.75294118\n",
      "  0.6627451  0.89019608 0.81568627 0.85490196 0.87843137 0.83137255\n",
      "  0.88627451 0.77254902 0.81960784 0.20392157]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.02352941 0.\n",
      "  0.38823529 0.95686275 0.87058824 0.8627451  0.85490196 0.79607843\n",
      "  0.77647059 0.86666667 0.84313725 0.83529412 0.87058824 0.8627451\n",
      "  0.96078431 0.46666667 0.65490196 0.21960784]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.01568627 0.         0.\n",
      "  0.21568627 0.9254902  0.89411765 0.90196078 0.89411765 0.94117647\n",
      "  0.90980392 0.83529412 0.85490196 0.8745098  0.91764706 0.85098039\n",
      "  0.85098039 0.81960784 0.36078431 0.        ]\n",
      " [0.         0.         0.00392157 0.01568627 0.02352941 0.02745098\n",
      "  0.00784314 0.         0.         0.         0.         0.\n",
      "  0.92941176 0.88627451 0.85098039 0.8745098  0.87058824 0.85882353\n",
      "  0.87058824 0.86666667 0.84705882 0.8745098  0.89803922 0.84313725\n",
      "  0.85490196 1.         0.30196078 0.        ]\n",
      " [0.         0.01176471 0.         0.         0.         0.\n",
      "  0.         0.         0.         0.24313725 0.56862745 0.8\n",
      "  0.89411765 0.81176471 0.83529412 0.86666667 0.85490196 0.81568627\n",
      "  0.82745098 0.85490196 0.87843137 0.8745098  0.85882353 0.84313725\n",
      "  0.87843137 0.95686275 0.62352941 0.        ]\n",
      " [0.         0.         0.         0.         0.07058824 0.17254902\n",
      "  0.32156863 0.41960784 0.74117647 0.89411765 0.8627451  0.87058824\n",
      "  0.85098039 0.88627451 0.78431373 0.80392157 0.82745098 0.90196078\n",
      "  0.87843137 0.91764706 0.69019608 0.7372549  0.98039216 0.97254902\n",
      "  0.91372549 0.93333333 0.84313725 0.        ]\n",
      " [0.         0.22352941 0.73333333 0.81568627 0.87843137 0.86666667\n",
      "  0.87843137 0.81568627 0.8        0.83921569 0.81568627 0.81960784\n",
      "  0.78431373 0.62352941 0.96078431 0.75686275 0.80784314 0.8745098\n",
      "  1.         1.         0.86666667 0.91764706 0.86666667 0.82745098\n",
      "  0.8627451  0.90980392 0.96470588 0.        ]\n",
      " [0.01176471 0.79215686 0.89411765 0.87843137 0.86666667 0.82745098\n",
      "  0.82745098 0.83921569 0.80392157 0.80392157 0.80392157 0.8627451\n",
      "  0.94117647 0.31372549 0.58823529 1.         0.89803922 0.86666667\n",
      "  0.7372549  0.60392157 0.74901961 0.82352941 0.8        0.81960784\n",
      "  0.87058824 0.89411765 0.88235294 0.        ]\n",
      " [0.38431373 0.91372549 0.77647059 0.82352941 0.87058824 0.89803922\n",
      "  0.89803922 0.91764706 0.97647059 0.8627451  0.76078431 0.84313725\n",
      "  0.85098039 0.94509804 0.25490196 0.28627451 0.41568627 0.45882353\n",
      "  0.65882353 0.85882353 0.86666667 0.84313725 0.85098039 0.8745098\n",
      "  0.8745098  0.87843137 0.89803922 0.11372549]\n",
      " [0.29411765 0.8        0.83137255 0.8        0.75686275 0.80392157\n",
      "  0.82745098 0.88235294 0.84705882 0.7254902  0.77254902 0.80784314\n",
      "  0.77647059 0.83529412 0.94117647 0.76470588 0.89019608 0.96078431\n",
      "  0.9372549  0.8745098  0.85490196 0.83137255 0.81960784 0.87058824\n",
      "  0.8627451  0.86666667 0.90196078 0.2627451 ]\n",
      " [0.18823529 0.79607843 0.71764706 0.76078431 0.83529412 0.77254902\n",
      "  0.7254902  0.74509804 0.76078431 0.75294118 0.79215686 0.83921569\n",
      "  0.85882353 0.86666667 0.8627451  0.9254902  0.88235294 0.84705882\n",
      "  0.78039216 0.80784314 0.72941176 0.70980392 0.69411765 0.6745098\n",
      "  0.70980392 0.80392157 0.80784314 0.45098039]\n",
      " [0.         0.47843137 0.85882353 0.75686275 0.70196078 0.67058824\n",
      "  0.71764706 0.76862745 0.8        0.82352941 0.83529412 0.81176471\n",
      "  0.82745098 0.82352941 0.78431373 0.76862745 0.76078431 0.74901961\n",
      "  0.76470588 0.74901961 0.77647059 0.75294118 0.69019608 0.61176471\n",
      "  0.65490196 0.69411765 0.82352941 0.36078431]\n",
      " [0.         0.         0.29019608 0.74117647 0.83137255 0.74901961\n",
      "  0.68627451 0.6745098  0.68627451 0.70980392 0.7254902  0.7372549\n",
      "  0.74117647 0.7372549  0.75686275 0.77647059 0.8        0.81960784\n",
      "  0.82352941 0.82352941 0.82745098 0.7372549  0.7372549  0.76078431\n",
      "  0.75294118 0.84705882 0.66666667 0.        ]\n",
      " [0.00784314 0.         0.         0.         0.25882353 0.78431373\n",
      "  0.87058824 0.92941176 0.9372549  0.94901961 0.96470588 0.95294118\n",
      "  0.95686275 0.86666667 0.8627451  0.75686275 0.74901961 0.70196078\n",
      "  0.71372549 0.71372549 0.70980392 0.69019608 0.65098039 0.65882353\n",
      "  0.38823529 0.22745098 0.         0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.15686275 0.23921569 0.17254902 0.28235294 0.16078431\n",
      "  0.1372549  0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.        ]\n",
      " [0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.         0.         0.\n",
      "  0.         0.         0.         0.        ]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAUFElEQVR4nO3da2yc1ZkH8P8z4/ElzjiJk+CE4BIuoZDCEqhJuIlSKDREVQOli4gQCxLaoF3otl0+gGhXZb+sEFpAaNntroEsYVWoWhUERREFzCULlDQmpOS2ITeHxDi2ExPbcTz2XJ794Bdqgs/zmnnnRs7/J1kezzNn5njGf78zc+acI6oKIjr+xcrdASIqDYadyBMMO5EnGHYiTzDsRJ6oKuWNVUuN1qK+lDdJ5JUUhjCqIzJRLVLYRWQpgEcAxAE8rqr3W5evRT2WyJVRbpKIDOu0zVnL+2m8iMQB/DuAawAsBLBCRBbme31EVFxRXrMvBrBTVXer6iiAXwNYXphuEVGhRQn7PAD7xv28Pzjvc0RkpYi0i0h7GiMRbo6Ioij6u/Gq2qqqLarakkBNsW+OiByihL0TQPO4n08KziOiChQl7OsBLBCRU0SkGsCNAF4oTLeIqNDyHnpT1YyI3AngDxgbelulqlsK1jMiKqhI4+yqugbAmgL1hYiKiB+XJfIEw07kCYadyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeYJhJ/IEw07kCYadyBMMO5EnGHYiT5R0KWkqA5lwVeG/iLixZ3xmo1n/5LtnOGsNT78b6bbDfjepSjhrmh6NdttRhT0uljwfMx7ZiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPcJz9OCfxuFnXTMasxxbZe3Vuu32q3X7YXUsMLTbbVg3nzHri5XazHmksPWwMP+R+hdjH0Sh9kyojtsbDySM7kScYdiJPMOxEnmDYiTzBsBN5gmEn8gTDTuQJjrMf58wxWYSPs+/77nSzftNF/2vW3+491VnbWzPHbKt1ZhlV37nIrJ/xH53OWqbjI/vKQ+aMh91vYeIzZriL2azZNjsw4C4a3Y4UdhHpADAIIAsgo6otUa6PiIqnEEf2b6vqwQJcDxEVEV+zE3kiatgVwMsi8p6IrJzoAiKyUkTaRaQ9jZGIN0dE+Yr6NP5SVe0UkRMAvCIi/6eqa8dfQFVbAbQCQIM0RlvdkIjyFunIrqqdwfceAM8BsKcxEVHZ5B12EakXkeSnpwFcDWBzoTpGRIUV5Wl8E4DnZGzebxWAp1X1pYL0igoml0pFaj963hGz/sNp9pzy2ljaWXszZs9X73yt2axn/8ru296Hks5a7v2LzbYzN9tj3Q3vd5n1g5fNM+u933S/om0KWU5/xqu7nDXpc0c677Cr6m4A5+bbnohKi0NvRJ5g2Ik8wbATeYJhJ/IEw07kCdGIW/Z+GQ3SqEvkypLdnjesZY9DHt8jN1xo1q/5+Rtm/azaj836YK7WWRvVaB/gfHT7t8z60O5pzlpsNGTL5JBytsleClrT9nF0xgb37163vNtsK4/NdtY+aHsER/r2Tdh7HtmJPMGwE3mCYSfyBMNO5AmGncgTDDuRJxh2Ik9wnL0ShGwPHEnI43v2e/b/+x/MsKewhokbaxsPabXZ9nC2PtJt92bcU1zTIWP8j++wp8AeMcbwASCWsR/Tq779vrN2feN6s+0Dp53jrK3TNgxoH8fZiXzGsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPcMvmSlDCzzoca8eRE8z6oYapZv1Axt7SeWbcvdxzMjZstp2fsPcL7c26x9EBIJ5wL1U9qnGz7T9/4/dmPXVWwqwnxF6K+mJjHYC/3vo3Ztt67DbrLjyyE3mCYSfyBMNO5AmGncgTDDuRJxh2Ik8w7ESe4Di752bX2Nse14p7y2UAqJaMWf84PcNZ2zH8dbPthwP2ZwCWNm0x62ljLN2aZw+Ej5OfmPjErKfUHoe37tVLmuxx9I1m1S30yC4iq0SkR0Q2jzuvUUReEZEdwXf3I0pEFWEyT+OfBLD0mPPuAdCmqgsAtAU/E1EFCw27qq4F0HfM2csBrA5OrwZwbYH7RUQFlu9r9iZV7QpOHwDQ5LqgiKwEsBIAajElz5sjoqgivxuvYytWOt/tUNVWVW1R1ZYEaqLeHBHlKd+wd4vIXAAIvvcUrktEVAz5hv0FALcEp28B8HxhukNExRL6ml1EngFwOYBZIrIfwC8A3A/gNyJyG4C9AG4oZiePeyHrxkvcnnutGfdYd3yGPSr6rembzHpvtsGsH87a78NMjx911gYz7r3bAaBv2L7uM2u6zPqGo/OdtdnV9ji51W8A6BidZdYX1Bww6w90u/dPaK499v3wz8tceZmzpuv+6KyFhl1VVzhK3O2B6CuEH5cl8gTDTuQJhp3IEww7kScYdiJPcIprJQhZSlqq7IfJGnrbd9tZZtsrpthLJr+TmmfWZ1cNmnVrmuncmn6zbbIpZdbDhv0aq9zTdwezdWbbKbERsx72e59fbS+D/dNXz3fWkmcfMts2JIxjtDGKyyM7kScYdiJPMOxEnmDYiTzBsBN5gmEn8gTDTuQJjrNXAElUm/Vcyh5vtszaNGrWD2btJY+nx+ypntUhSy5bWyNf3LjHbNsbMha+YfgUs56Mu7eEnh2zx8mbE/ZY96ZUs1lfM3S6Wb/te686a8+0XmW2rX7pHWdN1P148chO5AmGncgTDDuRJxh2Ik8w7ESeYNiJPMGwE3niqzXObiy5LFX2eLHEQ/6vxex6LmXMb87ZY81hNG2PhUfxyH89atb3Zaab9QNpux625HLWmGD97vA0s21tzN4uenbVgFkfyNnj9JbBnL3MtTVPHwjv+90zdzhrz/Z/x2ybLx7ZiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPVNQ4e5T10cPGqtUe9iyr4eWLzfq+a+1x/JvO+5OzdiCTNNu+b2xrDADTjDnhAFAfsr56St2ff/h41N5OOmys2loXHgBOMMbhs2of5zrTdt/ChH3+YH/GWNP++/Zc++lP5dWl8CO7iKwSkR4R2TzuvPtEpFNENgZfy/K7eSIqlck8jX8SwNIJzn9YVRcFX2sK2y0iKrTQsKvqWgB9JegLERVRlDfo7hSRD4Kn+c4XOCKyUkTaRaQ9Dfv1HREVT75h/yWA0wAsAtAF4EHXBVW1VVVbVLUlgZo8b46Iosor7KrarapZVc0BeAyA/XYyEZVdXmEXkbnjfrwOwGbXZYmoMoSOs4vIMwAuBzBLRPYD+AWAy0VkEQAF0AHg9kJ0xhpHj6pq7hyznj6lyaz3neXeC/zoHGNTbACLlm0z67c2/bdZ7802mPWEGPuzp2eabc+b0mHWX+tfaNYPVk0169Y4/cX17jndAHA4Z++/fmLVJ2b97p0/dNaapthj2Y+fbA8wpTVn1ren7Zes/Tn3fPh/WPi62fY5zDbrLqFhV9UVE5z9RF63RkRlw4/LEnmCYSfyBMNO5AmGncgTDDuRJypqiuvINReY9RN+tttZW9Sw32y7sO4ts57K2UtRW9Mttw7PM9sezdlbMu8YtYcF+zP2EFRc3MNAPaP2FNcH99jLFrct/k+z/vOPJ5oj9RexOnXWDmXtYbvrp9pLRQP2Y3b719Y6a6dW95htXxyaa9Y/DpkC25ToN+vzE73O2g+SH5pt8x1645GdyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeYJhJ/JEacfZxV4uesm/rDebX5nc4qwdVXtKYdg4eti4qWValb1s8Ejavpt70vYU1jBn1Bxw1q5r2Gi2XfvoErN+aepHZn3XFfb03LZh91TO3oz9e9+45wqzvuGjZrN+4fw9zto5yU6zbdhnG5LxlFm3ph0DwFDO/ff6bsr+/EG+eGQn8gTDTuQJhp3IEww7kScYdiJPMOxEnmDYiTwhqu75xoVWN6dZT7v5H5311jv+zWz/dN+Fzlpzrb0d3cnVB836zLi9/a8lGbPHXL+esMdcXxw6yay/cfhMs/7NZIezlhB7u+fLp+w067f+9C6znqm1l9EemO8+nmTq7b+9hnMPmfUfnf6aWa82fvfDWXscPex+C9uSOYy1BkEyZm+T/eCy65y1P3Y8if7hrgkfFB7ZiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPlHQ+eywNTOl2jy++OLDIbH9qnXut7YNpe330Pxw5x6yfVGdv/2ttPXy6MZ8cADamppv1l3q/YdZPrLPXT+9OT3PWDqXrzbZHjXnVAPDEww+Z9Qe77XXnr2vc4KydW22Pox/O2ceirSHr7Q/map21lNrrG/SHjMMnjb8HAEirHa24seXz9Jg9hj9wjnsb7my3+3ZDj+wi0iwir4vIVhHZIiI/Ds5vFJFXRGRH8D3/1R+IqOgm8zQ+A+AuVV0I4EIAd4jIQgD3AGhT1QUA2oKfiahChYZdVbtUdUNwehDANgDzACwHsDq42GoA1xark0QU3Zd6g05E5gM4D8A6AE2q2hWUDgBocrRZKSLtItKeGRmK0FUiimLSYReRqQB+B+Anqvq5d4x0bDbNhLMaVLVVVVtUtaWqxn6ziIiKZ1JhF5EExoL+K1V9Nji7W0TmBvW5AOxtMYmorEKH3kREADwBYJuqjh+HeQHALQDuD74/H3Zd8dEckvtGnPWc2tMlXzvonurZVDtotl2U3GfWtx+1h3E2DZ/orG2o+prZti7u3u4ZAKZV21Nk66vc9xkAzEq4f/dTauz/wdY0UABYn7J/t7+b/YZZ/yjjHqT5/dAZZtutR933OQDMCFnCe9OAu/3RjL2N9kjWjkYqYw/lTquxH9MLGvc6a9thbxfde64xbfhtd7vJjLNfAuBmAJtE5NNFyO/FWMh/IyK3AdgL4IZJXBcRlUlo2FX1LQCuQ+6Vhe0OERULPy5L5AmGncgTDDuRJxh2Ik8w7ESeKO2WzUeGEXvzfWf5ty9fYjb/p+W/ddbeDFlu+cUD9rjowKg91XP2FPdHfRuMcW4AaEzYHxMO2/K5NmT7308y7k8mjsTsqZxZ50DLmAMj7umzAPB2boFZT+fcWzaPGDUg/PMJfaOzzPqJdf3O2mDGPf0VADoGG836wX57W+XUFDtab2VPc9aWznFvTQ4AdT3uxyxm/KnwyE7kCYadyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeaKkWzY3SKMukfwnyvXf5N6y+dS/3262XTx9j1nfMGDP2/7IGHdNhyx5nIi5lw0GgCmJUbNeGzLeXB13z0mPTbyA0GdyIePs9XG7b2Fz7Ruq3PO6k3F7znfM2NZ4MuLG7/6n/vmRrjsZ8ntn1P6buGjaLmdt1Z6LzbbTlrm32V6nbRjQPm7ZTOQzhp3IEww7kScYdiJPMOxEnmDYiTzBsBN5ovTj7PGr3RfI2WuYRzF0/RKzvuTe9XY96R4XPbO622ybgD1eXBsynlwfs8fCU8ZjGPbf/K3hZrOeDbmG1z45y6ynjfHm7qMNZtuE8fmBybD2IRjOhGzZPGzPd4/H7Nyk3rDn2s/c6v7sRM0a+2/RwnF2ImLYiXzBsBN5gmEn8gTDTuQJhp3IEww7kSdCx9lFpBnAUwCaACiAVlV9RETuA/C3AHqDi96rqmus64o6n71SyQX2mvTDc+rMes0he2704Ml2+4Zd7nXpYyP2mvO5P28z6/TVYo2zT2aTiAyAu1R1g4gkAbwnIq8EtYdV9V8L1VEiKp7J7M/eBaArOD0oItsAzCt2x4iosL7Ua3YRmQ/gPADrgrPuFJEPRGSViMxwtFkpIu0i0p6G/XSViIpn0mEXkakAfgfgJ6o6AOCXAE4DsAhjR/4HJ2qnqq2q2qKqLQnY+6kRUfFMKuwiksBY0H+lqs8CgKp2q2pWVXMAHgOwuHjdJKKoQsMuIgLgCQDbVPWhcefPHXex6wBsLnz3iKhQJvNu/CUAbgawSUQ2BufdC2CFiCzC2HBcB4Dbi9LDrwBdv8ms25MlwzW8k3/baIsx0/FkMu/GvwVMuLi4OaZORJWFn6Aj8gTDTuQJhp3IEww7kScYdiJPMOxEnmDYiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPMOxEnijpls0i0gtg77izZgE4WLIOfDmV2rdK7RfAvuWrkH07WVVnT1Qoadi/cOMi7araUrYOGCq1b5XaL4B9y1ep+san8USeYNiJPFHusLeW+fYtldq3Su0XwL7lqyR9K+trdiIqnXIf2YmoRBh2Ik+UJewislREtovIThG5pxx9cBGRDhHZJCIbRaS9zH1ZJSI9IrJ53HmNIvKKiOwIvk+4x16Z+nafiHQG991GEVlWpr41i8jrIrJVRLaIyI+D88t63xn9Ksn9VvLX7CISB/AhgKsA7AewHsAKVd1a0o44iEgHgBZVLfsHMETkMgBHADylqmcH5z0AoE9V7w/+Uc5Q1bsrpG/3AThS7m28g92K5o7fZhzAtQBuRRnvO6NfN6AE91s5juyLAexU1d2qOgrg1wCWl6EfFU9V1wLoO+bs5QBWB6dXY+yPpeQcfasIqtqlqhuC04MAPt1mvKz3ndGvkihH2OcB2Dfu5/2orP3eFcDLIvKeiKwsd2cm0KSqXcHpAwCaytmZCYRu411Kx2wzXjH3XT7bn0fFN+i+6FJVPR/ANQDuCJ6uViQdew1WSWOnk9rGu1Qm2Gb8M+W87/Ld/jyqcoS9E0DzuJ9PCs6rCKraGXzvAfAcKm8r6u5Pd9ANvveUuT+fqaRtvCfaZhwVcN+Vc/vzcoR9PYAFInKKiFQDuBHAC2XoxxeISH3wxglEpB7A1ai8rahfAHBLcPoWAM+XsS+fUynbeLu2GUeZ77uyb3+uqiX/ArAMY+/I7wLws3L0wdGvUwH8OfjaUu6+AXgGY0/r0hh7b+M2ADMBtAHYAeBVAI0V1Lf/AbAJwAcYC9bcMvXtUow9Rf8AwMbga1m57zujXyW53/hxWSJP8A06Ik8w7ESeYNiJPMGwE3mCYSfyBMNO5AmGncgT/w8K8iUImXY9pQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.imshow(training_images[0])\n",
    "print(training_labels[0])\n",
    "print(training_images[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3cbrdH225_nH"
   },
   "source": [
    "You'll notice that all of the values in the number are between 0 and 255. If we are training a neural network, for various reasons it's easier if we treat all values as between 0 and 1, a process called '**normalizing**'...and fortunately in Python it's easy to normalize a list like this without looping. You do it like this:\n",
    "\n",
    "你会注意到，数字中的所有值都在0和255之间。如果我们要训练一个神经网络，出于多种原因，如果把所有的值都处理成0和1之间，那就更容易得到较好的训练效果。这个过程叫做 \"**归一化**\"......幸运的是，在Python中，很容易对这样的列表进行归一化，而不需要循环。可以这样做："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kRH19pWs6ZDn"
   },
   "outputs": [],
   "source": [
    "training_images  = training_images / 255.0\n",
    "test_images = test_images / 255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3DkO0As46lRn"
   },
   "source": [
    "Now you might be wondering why there are 2 sets...training and testing -- remember we spoke about this in the intro? The idea is to have 1 set of data for training, and then another set of data...that the model hasn't yet seen...to see how good it would be at classifying values. After all, when you're done, you're going to want to try it out with data that it hadn't previously seen!\n",
    "\n",
    "你可能在想为什么有2组数据-训练集和测试集。记得在介绍中说过的吗？基本想法是将1组数据用于训练，然后用另一组数据评估模型在分类值方面的表现会有多好。测试数据必须是模型还没有看到过的。毕竟，当完成模型的训练后，必定想用它之前没有见过的数据来试一试!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dIn7S9gf62ie"
   },
   "source": [
    "Let's now design the model. There's quite a few new concepts here, but don't worry, you'll get the hang of them. \n",
    "现在我们来设计Fashion MNIST分类模型。这里有不少新的概念，不过别担心，你会掌握它们的窍门。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7mAyndG3kVlK"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), \n",
    "                                    tf.keras.layers.Dense(128, activation=tf.nn.relu), \n",
    "                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-lUcWaiX7MFj"
   },
   "source": [
    "**Sequential**: That defines a SEQUENCE of layers in the neural network\n",
    "\n",
    "**Flatten**: Remember earlier where our images were a square, when you printed them out? Flatten just takes that square and turns it into a 1 dimensional set.\n",
    "\n",
    "**Dense**: Adds a layer of neurons\n",
    "\n",
    "Each layer of neurons need an **activation function** to tell them what to do. There's lots of options, but just use these for now. \n",
    "\n",
    "**Relu** effectively means \"If X>0 return X, else return 0\" -- so what it does it it only passes values 0 or greater to the next layer in the network.\n",
    "\n",
    "**Softmax** takes a set of values, and effectively picks the biggest one, so, for example, if the output of the last layer looks like [0.1, 0.1, 0.05, 0.1, 9.5, 0.1, 0.05, 0.05, 0.05], it saves you from fishing through it looking for the biggest value, and turns it into [0,0,0,0,1,0,0,0,0] -- The goal is to save a lot of coding!\n",
    "\n",
    "**Sequential**。这定义了神经网络中的层数序列。一开始学习神经元网络总是使用序列模型。\n",
    "\n",
    "**Flatten**。还记得上面将图像打印出来的时候是一个正方形吗？扁平化只是把这个正方形变成了一个一维的集合。把二维数组变成一维数组。\n",
    "\n",
    "**Dense**：增加一层神经元。\n",
    "\n",
    "每一层神经元都需要一个**激活函数 activation**来告诉它们输出什么。有很多选项，但目前只用这些（relu和softmax）。\n",
    "\n",
    "**Relu**的意思是 \"如果X>0返回X，否则返回0\"--所以它的作用是它只把大于0的值传递给网络中的下一层，小于0的也当作0。\n",
    "\n",
    "**Softmax**激活函数接收到一组值后，选择其中最大的一个输出。例如，上一层的输出为[0.1, 0.1, 0.05, 0.1, 9.5, 0.1, 0.05, 0.05, 0.05]，Softmax就省去了你在其中寻找最大的值，并把它变成[0,0,0,0,0,1,0,0,0,0,0] **Softmax**的意思是 \"如果X>0，则返回X，否则返回0\" -- 所以它的作用是只把0或更大的值传给下一层的网络。--其目的是节省大量的编码！\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c8vbMCqb9Mh6"
   },
   "source": [
    "The next thing to do, now the model is defined, is to actually build it. You do this by compiling it with an optimizer and loss function as before -- and then you train it by calling **model.fit ** asking it to fit your training data to your training labels -- i.e. have it figure out the relationship between the training data and its actual labels, so in future if you have data that looks like the training data, then it can make a prediction for what that data would look like. \n",
    "\n",
    "现在模型已经定义好了，下一步要做的事情就是实际建立它。可以像之前一样用优化器和损失函数编译它--然后通过调用*model.fit*来训练它，要求它将训练数据与标签拟合--即让模型找出训练数据和标签之间的关系。训练好之后，它将能对格式与训练数据相同，但从未“见过”的新数据做出预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 29548,
     "status": "ok",
     "timestamp": 1550587849991,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "BLMdl9aP8nQ0",
    "outputId": "91a80606-f0a5-41f0-9d03-91d1ecdc8e17"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 1.0914 - accuracy: 0.6527\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.6428 - accuracy: 0.7702\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.5682 - accuracy: 0.7957\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.5264 - accuracy: 0.8123\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.4993 - accuracy: 0.8235\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fab31e288d0>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.compile(optimizer = tf.optimizers.Adam(),\n",
    "              loss = 'sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.fit(training_images, training_labels, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-JJMsvSB-1UY"
   },
   "source": [
    "Once it's done training -- you should see an accuracy value at the end of the final epoch. It might look something like 0.9098. This tells you that your neural network is about 91% accurate in classifying the training data. I.E., it figured out a pattern match between the image and the labels that worked 91% of the time. Not great, but not bad considering it was only trained for 5 epochs and done quite quickly.\n",
    "\n",
    "But how would it work with unseen data? That's why we have the test images. We can call model.evaluate, and pass in the two sets, and it will report back the loss for each. Let's give it a try:\n",
    "\n",
    "一旦模型完成了训练 -- 能在最后一个epoch结束时看到一个准确率值。它可能看起来像0.9098。这告诉我们，神经网络对训练数据的分类准确率约为91%。即，它找出了图像和标签之间的模式匹配，91%的分类结果都正确。考虑到只训练了5个epochs，而且做得相当快，所以结果还不错。\n",
    "\n",
    "但对于未见过的数据，它的分类准确度有多高？这就是为什么我们需要测试图像的原因。我们可以调用model.evaluation，并将用于测试的图像和标签数据传入，它会报告测试数据的损失。让我们试一试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 832,
     "status": "ok",
     "timestamp": 1550587895553,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "WzlqsEzX9s5P",
    "outputId": "4dd8018a-f3ba-4e5b-e163-3d4f5dd12f19"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 0s 836us/step - loss: 0.5118 - accuracy: 0.8162\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.5118343830108643, 0.8162000179290771]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6tki-Aro_Uax"
   },
   "source": [
    "For me, that returned a accuracy of about .8838, which means it was about 88% accurate. As expected it probably would not do as well with *unseen* data as it did with data it was trained on!  As you go through this course, you'll look at ways to improve this. \n",
    "\n",
    "To explore further, try the below exercises:\n",
    "\n",
    "模型评估返回的准确率约为0.8838，这意味着它的准确率约为88%。正如预期的那样，模型对*未见过*的数据可能不会像对它训练过的数据那样做得那么好!  在这门课程中，我们会学习如何提高模型对测试数据的准确率。\n",
    "\n",
    "请尝试下面的练习，进一步探索。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "htldZNWcIPSN"
   },
   "source": [
    "# Exploration Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rquQqIx4AaGR"
   },
   "source": [
    "###Exercise 1:\n",
    "For this first exercise run the below code: It creates a set of classifications for each of the test images, and then prints the first entry in the classifications. The output, after you run it is a list of numbers. Why do you think this is, and what do those numbers represent? \n",
    "\n",
    "###练习1\n",
    "请运行以下代码。程序根据测试图像集生成了一组分类，然后打印出分类中的第一条记录。 运行print后，输出的是一个数字列表。请解释下这些数字代表什么？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RyEIki0z_hAD"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3.6924187e-06 1.7662140e-07 1.3648037e-05 1.0854434e-05 2.1243884e-05\n",
      " 1.3200106e-01 3.4377190e-05 3.6959159e-01 5.9984685e-03 4.9232492e-01]\n"
     ]
    }
   ],
   "source": [
    "classifications = model.predict(test_images)\n",
    "\n",
    "print(classifications[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MdzqbQhRArzm"
   },
   "source": [
    "Hint: try running print(test_labels[0]) -- and you'll get a 9. Does that help you understand why this list looks the way it does? \n",
    "\n",
    "提示：试着运行print(test_labels[0]) -- 会得到一个9。这是否有助于理解为什么列表看起来是这样的？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WnBGOrMiA1n5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n"
     ]
    }
   ],
   "source": [
    "print(test_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uUs7eqr7uSvs"
   },
   "source": [
    "### What does this list represent?\n",
    "\n",
    "\n",
    "1.   It's 10 random meaningless values\n",
    "2.   It's the first 10 classifications that the computer made\n",
    "3.   It's the probability that this item is each of the 10 classes\n",
    "\n",
    "###这个列表代表什么？\n",
    "\n",
    "1.   这是10个随机的无意义的值\n",
    "2.   这是计算机做出的前10张图片的分类。\n",
    "3.   这是一张图像对应10类中每一类的概率。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wAbr92RTA67u"
   },
   "source": [
    "####Answer: \n",
    "The correct answer is (3)\n",
    "\n",
    "The output of the model is a list of 10 numbers. These numbers are a probability that the value being classified is the corresponding value, i.e. the first value in the list is the probability that the handwriting is of a '0', the next is a '1' etc. Notice that they are all VERY LOW probabilities.\n",
    "\n",
    "For the 7, the probability was .999+, i.e. the neural network is telling us that it's almost certainly a 7.\n",
    "\n",
    "####Answer。\n",
    "正确答案是(3)\n",
    "\n",
    "该模型的输出是一个由10个数字组成的列表。这些数字是被分类的值与对应值的概率，即列表中的第一个值是类别 \"0 \"的概率，下一个是 \"1 \"的概率，等等。注意，它们的概率都很低。\n",
    "\n",
    "对于9，概率最高，即神经网络告诉我们，最可能的是9。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CD4kC6TBu-69"
   },
   "source": [
    "### How do you know that this list tells you that the item is an ankle boot?\n",
    "\n",
    "\n",
    "1.   There's not enough information to answer that question\n",
    "2.   The 10th element on the list is the biggest, and the ankle boot is labelled 9\n",
    "2.   The ankle boot is label 9, and there are 0->9 elements in the list\n",
    "\n",
    "###怎么知道这个列表告诉你这个商品是踝靴？\n",
    "\n",
    "\n",
    "1.   没有足够的信息来回答这个问题。\n",
    "2.   列表中的第10个元素是最大的，并且踝靴的标签是9。\n",
    "2.   踝靴的标签为9，列表中存在0->9个元素。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I-haLncrva5L"
   },
   "source": [
    "####Answer\n",
    "The correct answer is (2). Both the list and the labels are 0 based, so the ankle boot having label 9 means that it is the 10th of the 10 classes. The list having the 10th element being the highest value means that the Neural Network has predicted that the item it is classifying is most likely an ankle boot\n",
    "\n",
    "#### 回答\n",
    "正确答案是（2）。列表和标签都是以0为基础的，所以脚踝靴的标签为9，意味着它是10个类别中的第10个。列表中第10个元素的值最高，意味着神经网络已经预测出它所分类的物品最有可能是一只脚踝靴。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OgQSIfDSOWv6"
   },
   "source": [
    "##Exercise 2: \n",
    "Let's now look at the layers in your model. Experiment with different values for the dense layer with 512 neurons. What different results do you get for loss, training time etc? Why do you think that's the case? \n",
    "\n",
    "##练习2。\n",
    "现在让我们看看模型的中间层。尝试用不同值代替512（神经元），对全连接层进行实验。得到的损失、训练时间等结果有什么不同？为什么会这样？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GSZSwV5UObQP"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 5s 3ms/step - loss: 0.1854\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 5s 3ms/step - loss: 0.0764\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 5s 3ms/step - loss: 0.0487\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 5s 3ms/step - loss: 0.0345\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 5s 3ms/step - loss: 0.0258\n",
      "313/313 [==============================] - 0s 1ms/step - loss: 0.0820\n",
      "[5.44085501e-13 2.37001058e-10 1.47267731e-09 2.23942280e-07\n",
      " 1.81040045e-14 4.23169677e-09 6.72709204e-15 9.99999762e-01\n",
      " 1.15229375e-10 3.61739012e-08]\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(training_images, training_labels) ,  (test_images, test_labels) = mnist.load_data()\n",
    "\n",
    "training_images = training_images/255.0\n",
    "test_images = test_images/255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
    "                                    tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
    "                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "              loss = 'sparse_categorical_crossentropy')\n",
    "\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "\n",
    "model.evaluate(test_images, test_labels)\n",
    "\n",
    "classifications = model.predict(test_images)\n",
    "\n",
    "print(classifications[0])\n",
    "print(test_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bOOEnHZFv5cS"
   },
   "source": [
    "###Question 1. Increase to 1024 Neurons -- What's the impact?\n",
    "\n",
    "1. Training takes longer, but is more accurate\n",
    "2. Training takes longer, but no impact on accuracy\n",
    "3. Training takes the same time, but is more accurate\n",
    "\n",
    "###问题1：增加到1024个神经元--有什么影响？\n",
    "\n",
    "1. 训练时间更长，但更准确\n",
    "2. 训练时间较长，但对准确性没有影响\n",
    "3. 训练需要同样的时间，但更准确。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "U73MUP2lwrI2"
   },
   "source": [
    "####Answer\n",
    "The correct answer is (1) by adding more Neurons we have to do more calculations, slowing down the process, but in this case they have a good impact -- we do get more accurate. That doesn't mean it's always a case of 'more is better', you can hit the law of diminishing returns very quickly!\n",
    "\n",
    "#### 回答\n",
    "正确的答案是(1)通过增加更多的神经元，计算机必须做更多的计算，减慢了训练过程。但对于本案例，增加神经元数量有积极的影响--确实得到了更好的准确度。但这并不意味着总是 \"越多越好\"，因为很快就会遇到收益递减的定律。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WtWxK16hQxLN"
   },
   "source": [
    "##Exercise 3: \n",
    "\n",
    "What would happen if you remove the Flatten() layer. Why do you think that's the case? \n",
    "\n",
    "You get an error about the shape of the data. It may seem vague right now, but it reinforces the rule of thumb that the first layer in your network should be the same shape as your data. Right now our data is 28x28 images, and 28 layers of 28 neurons would be infeasible, so it makes more sense to 'flatten' that 28,28 into a 784x1. Instead of wriitng all the code to handle that ourselves, we add the Flatten() layer at the begining, and when the arrays are loaded into the model later, they'll automatically be flattened for us.\n",
    "\n",
    "##练习3。\n",
    "\n",
    "如果删除Flatten()层，会发生什么情况？为什么会出现这种情况？\n",
    "\n",
    "会得到一个关于数据形状的错误。因为神经元网络中的第一层应该与输入数据的形状相同。现在我们的数据是28x28的图像，28层28个神经元是不可行的，所以把28,28 \"扁平化 \"成784x1。不需要自己编写所有的代码来实现扁平化，而是在模型最上面添加Flatten()层，当把数组加载到模型中时，它们会自动为扁平化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "collapsed": true,
    "id": "ExNxCwhcQ18S",
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "in user code:\n\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:571 train_function  *\n        outputs = self.distribute_strategy.run(\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **\n        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica\n        return self._call_for_each_replica(fn, args, kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica\n        return fn(*args, **kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:533 train_step  **\n        y, y_pred, sample_weight, regularization_losses=self.losses)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/compile_utils.py:205 __call__\n        loss_value = loss_obj(y_t, y_p, sample_weight=sw)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:143 __call__\n        losses = self.call(y_true, y_pred)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:246 call\n        return self.fn(y_true, y_pred, **self._fn_kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:1558 sparse_categorical_crossentropy\n        y_true, y_pred, from_logits=from_logits, axis=axis)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/backend.py:4655 sparse_categorical_crossentropy\n        labels=target, logits=output)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py:3591 sparse_softmax_cross_entropy_with_logits_v2\n        labels=labels, logits=logits, name=name)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py:3507 sparse_softmax_cross_entropy_with_logits\n        logits.get_shape()))\n\n    ValueError: Shape mismatch: The shape of labels (received (32, 1)) should equal the shape of logits except for the last dimension (received (32, 28, 10)).\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-cd0411f1d295>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     16\u001b[0m               loss = 'sparse_categorical_crossentropy')\n\u001b[1;32m     17\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 18\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtraining_images\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtraining_labels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     19\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     20\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mevaluate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_images\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_labels\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36m_method_wrapper\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m     64\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_method_wrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     65\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_in_multi_worker_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 66\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     67\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     68\u001b[0m     \u001b[0;31m# Running inside `run_distribute_coordinator` already.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[1;32m    846\u001b[0m                 batch_size=batch_size):\n\u001b[1;32m    847\u001b[0m               \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 848\u001b[0;31m               \u001b[0mtmp_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    849\u001b[0m               \u001b[0;31m# Catch OutOfRangeError for Datasets of unknown size.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    850\u001b[0m               \u001b[0;31m# This blocks until the batch has finished executing.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    578\u001b[0m         \u001b[0mxla_context\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mExit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    579\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 580\u001b[0;31m       \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    581\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    582\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mtracing_count\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_tracing_count\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    625\u001b[0m       \u001b[0;31m# This is the first call of __call__, so we have to initialize.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    626\u001b[0m       \u001b[0minitializers\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 627\u001b[0;31m       \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_initialize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0madd_initializers_to\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minitializers\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    628\u001b[0m     \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    629\u001b[0m       \u001b[0;31m# At this point we know that the initialization is complete (or less\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_initialize\u001b[0;34m(self, args, kwds, add_initializers_to)\u001b[0m\n\u001b[1;32m    504\u001b[0m     self._concrete_stateful_fn = (\n\u001b[1;32m    505\u001b[0m         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access\n\u001b[0;32m--> 506\u001b[0;31m             *args, **kwds))\n\u001b[0m\u001b[1;32m    507\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    508\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0minvalid_creator_scope\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0munused_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0munused_kwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_get_concrete_function_internal_garbage_collected\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   2444\u001b[0m       \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2445\u001b[0m     \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2446\u001b[0;31m       \u001b[0mgraph_function\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_maybe_define_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2447\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2448\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_maybe_define_function\u001b[0;34m(self, args, kwargs)\u001b[0m\n\u001b[1;32m   2775\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2776\u001b[0m       \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_function_cache\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmissed\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcall_context_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2777\u001b[0;31m       \u001b[0mgraph_function\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_create_graph_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2778\u001b[0m       \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_function_cache\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprimary\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mcache_key\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2779\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_create_graph_function\u001b[0;34m(self, args, kwargs, override_flat_arg_shapes)\u001b[0m\n\u001b[1;32m   2665\u001b[0m             \u001b[0marg_names\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0marg_names\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2666\u001b[0m             \u001b[0moverride_flat_arg_shapes\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0moverride_flat_arg_shapes\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2667\u001b[0;31m             capture_by_value=self._capture_by_value),\n\u001b[0m\u001b[1;32m   2668\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_function_attributes\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2669\u001b[0m         \u001b[0;31m# Tell the ConcreteFunction to clean up its graph once it goes out of\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py\u001b[0m in \u001b[0;36mfunc_graph_from_py_func\u001b[0;34m(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)\u001b[0m\n\u001b[1;32m    979\u001b[0m         \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moriginal_func\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_decorator\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munwrap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpython_func\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    980\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 981\u001b[0;31m       \u001b[0mfunc_outputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpython_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mfunc_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mfunc_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    982\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    983\u001b[0m       \u001b[0;31m# invariant: `func_outputs` contains only Tensors, CompositeTensors,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36mwrapped_fn\u001b[0;34m(*args, **kwds)\u001b[0m\n\u001b[1;32m    439\u001b[0m         \u001b[0;31m# __wrapped__ allows AutoGraph to swap in a converted function. We give\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    440\u001b[0m         \u001b[0;31m# the function a weak reference to itself to avoid a reference cycle.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 441\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mweak_wrapped_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__wrapped__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    442\u001b[0m     \u001b[0mweak_wrapped_fn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mweakref\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mref\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mwrapped_fn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    443\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    966\u001b[0m           \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# pylint:disable=broad-except\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    967\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"ag_error_metadata\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 968\u001b[0;31m               \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mag_error_metadata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    969\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    970\u001b[0m               \u001b[0;32mraise\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: in user code:\n\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:571 train_function  *\n        outputs = self.distribute_strategy.run(\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **\n        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica\n        return self._call_for_each_replica(fn, args, kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica\n        return fn(*args, **kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:533 train_step  **\n        y, y_pred, sample_weight, regularization_losses=self.losses)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/compile_utils.py:205 __call__\n        loss_value = loss_obj(y_t, y_p, sample_weight=sw)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:143 __call__\n        losses = self.call(y_true, y_pred)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:246 call\n        return self.fn(y_true, y_pred, **self._fn_kwargs)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:1558 sparse_categorical_crossentropy\n        y_true, y_pred, from_logits=from_logits, axis=axis)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/backend.py:4655 sparse_categorical_crossentropy\n        labels=target, logits=output)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py:3591 sparse_softmax_cross_entropy_with_logits_v2\n        labels=labels, logits=logits, name=name)\n    /opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py:3507 sparse_softmax_cross_entropy_with_logits\n        logits.get_shape()))\n\n    ValueError: Shape mismatch: The shape of labels (received (32, 1)) should equal the shape of logits except for the last dimension (received (32, 28, 10)).\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(training_images, training_labels) ,  (test_images, test_labels) = mnist.load_data()\n",
    "\n",
    "training_images = training_images/255.0\n",
    "test_images = test_images/255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([#tf.keras.layers.Flatten(),\n",
    "                                    tf.keras.layers.Dense(64, activation=tf.nn.relu),\n",
    "                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "              loss = 'sparse_categorical_crossentropy')\n",
    "\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "\n",
    "model.evaluate(test_images, test_labels)\n",
    "\n",
    "classifications = model.predict(test_images)\n",
    "\n",
    "print(classifications[0])\n",
    "print(test_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VqoCR-ieSGDg"
   },
   "source": [
    "##Exercise 4: \n",
    "\n",
    "Consider the final (output) layers. Why are there 10 of them? What would happen if you had a different amount than 10? For example, try training the network with 5\n",
    "\n",
    "You get an error as soon as it finds an unexpected value. Another rule of thumb -- the number of neurons in the last layer should match the number of classes you are classifying for. In this case it's the digits 0-9, so there are 10 of them, hence you should have 10 neurons in your final layer.\n",
    "\n",
    "##练习4。\n",
    "\n",
    "考虑最后（产出）层。为什么有10个神经元？如果数量少于10会发生什么？例如，尝试改作5个来训练网络。\n",
    "\n",
    "一旦模型发现一个意外的值，就会产生一个错误。规则是--最后一层的神经元数量应该与你要分类的类数相匹配。在这种情况下，是数字0-9，所以有10个，因此你的最后一层应该有10个神经元。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MMckVntcSPvo"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "ename": "InvalidArgumentError",
     "evalue": " Received a label value of 9 which is outside the valid range of [0, 5).  Label values: 8 3 8 9 6 6 0 5 7 6 2 7 8 2 1 7 9 3 3 6 0 0 2 1 8 7 4 6 5 3 0 0\n\t [[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at <ipython-input-19-04e5ea87fa1d>:18) ]] [Op:__inference_train_function_31729]\n\nFunction call stack:\ntrain_function\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-04e5ea87fa1d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     16\u001b[0m               loss = 'sparse_categorical_crossentropy')\n\u001b[1;32m     17\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 18\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtraining_images\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtraining_labels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     19\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     20\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mevaluate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_images\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_labels\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36m_method_wrapper\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m     64\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_method_wrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     65\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_in_multi_worker_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 66\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     67\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     68\u001b[0m     \u001b[0;31m# Running inside `run_distribute_coordinator` already.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[1;32m    846\u001b[0m                 batch_size=batch_size):\n\u001b[1;32m    847\u001b[0m               \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 848\u001b[0;31m               \u001b[0mtmp_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    849\u001b[0m               \u001b[0;31m# Catch OutOfRangeError for Datasets of unknown size.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    850\u001b[0m               \u001b[0;31m# This blocks until the batch has finished executing.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    578\u001b[0m         \u001b[0mxla_context\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mExit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    579\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 580\u001b[0;31m       \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    581\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    582\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mtracing_count\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_tracing_count\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    642\u001b[0m         \u001b[0;31m# Lifting succeeded, so variables are initialized and we can run the\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    643\u001b[0m         \u001b[0;31m# stateless function.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 644\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateless_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    645\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    646\u001b[0m       \u001b[0mcanon_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcanon_kwds\u001b[0m \u001b[0;34m=\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   2418\u001b[0m     \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2419\u001b[0m       \u001b[0mgraph_function\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_maybe_define_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2420\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_filtered_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2421\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2422\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_filtered_call\u001b[0;34m(self, args, kwargs)\u001b[0m\n\u001b[1;32m   1663\u001b[0m          if isinstance(t, (ops.Tensor,\n\u001b[1;32m   1664\u001b[0m                            resource_variable_ops.BaseResourceVariable))),\n\u001b[0;32m-> 1665\u001b[0;31m         self.captured_inputs)\n\u001b[0m\u001b[1;32m   1666\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1667\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_call_flat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcaptured_inputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcancellation_manager\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[0;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[1;32m   1744\u001b[0m       \u001b[0;31m# No tape is watching; skip to running the function.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1745\u001b[0m       return self._build_call_outputs(self._inference_function.call(\n\u001b[0;32m-> 1746\u001b[0;31m           ctx, args, cancellation_manager=cancellation_manager))\n\u001b[0m\u001b[1;32m   1747\u001b[0m     forward_backward = self._select_forward_and_backward_functions(\n\u001b[1;32m   1748\u001b[0m         \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36mcall\u001b[0;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[1;32m    596\u001b[0m               \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    597\u001b[0m               \u001b[0mattrs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mattrs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 598\u001b[0;31m               ctx=ctx)\n\u001b[0m\u001b[1;32m    599\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    600\u001b[0m           outputs = execute.execute_with_cancellation(\n",
      "\u001b[0;32m/opt/tljh/user/lib/python3.7/site-packages/tensorflow/python/eager/execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[1;32m     58\u001b[0m     \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     59\u001b[0m     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n\u001b[0;32m---> 60\u001b[0;31m                                         inputs, attrs, num_outputs)\n\u001b[0m\u001b[1;32m     61\u001b[0m   \u001b[0;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     62\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mInvalidArgumentError\u001b[0m:  Received a label value of 9 which is outside the valid range of [0, 5).  Label values: 8 3 8 9 6 6 0 5 7 6 2 7 8 2 1 7 9 3 3 6 0 0 2 1 8 7 4 6 5 3 0 0\n\t [[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at <ipython-input-19-04e5ea87fa1d>:18) ]] [Op:__inference_train_function_31729]\n\nFunction call stack:\ntrain_function\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(training_images, training_labels) ,  (test_images, test_labels) = mnist.load_data()\n",
    "\n",
    "training_images = training_images/255.0\n",
    "test_images = test_images/255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
    "                                    tf.keras.layers.Dense(64, activation=tf.nn.relu),\n",
    "                                    tf.keras.layers.Dense(5, activation=tf.nn.softmax)])\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "              loss = 'sparse_categorical_crossentropy')\n",
    "\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "\n",
    "model.evaluate(test_images, test_labels)\n",
    "\n",
    "classifications = model.predict(test_images)\n",
    "\n",
    "print(classifications[0])\n",
    "print(test_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-0lF5MuvSuZF"
   },
   "source": [
    "##Exercise 5: \n",
    "\n",
    "Consider the effects of additional layers in the network. What will happen if you add another layer between the one with 512 and the final layer with 10. \n",
    "\n",
    "Ans: There isn't a significant impact -- because this is relatively simple data. For far more complex data (including color images to be classified as flowers that you'll see in the next lesson), extra layers are often necessary. \n",
    "\n",
    "##练习5。\n",
    "\n",
    "考虑网络中增加层数的影响。如果在512层和10层之间再加一层会发生什么？\n",
    "\n",
    "答案：没有显著影响--因为这是相对简单的数据。对于复杂得多的数据，通常要增加额外的层。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "b1YPa6UhS8Es"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 4s 2ms/step - loss: 0.1850\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 4s 2ms/step - loss: 0.0808\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 4s 2ms/step - loss: 0.0550\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 4s 2ms/step - loss: 0.0412\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 4s 2ms/step - loss: 0.0330\n",
      "313/313 [==============================] - 0s 1ms/step - loss: 0.0769\n",
      "[8.0844446e-13 6.1690521e-08 6.4078776e-08 3.7278909e-07 2.6953151e-09\n",
      " 5.7083088e-10 2.3449347e-13 9.9999404e-01 5.7558780e-10 5.3824215e-06]\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(training_images, training_labels) ,  (test_images, test_labels) = mnist.load_data()\n",
    "\n",
    "training_images = training_images/255.0\n",
    "test_images = test_images/255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
    "                                    tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
    "                                    tf.keras.layers.Dense(256, activation=tf.nn.relu),\n",
    "                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "              loss = 'sparse_categorical_crossentropy')\n",
    "\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "\n",
    "model.evaluate(test_images, test_labels)\n",
    "\n",
    "classifications = model.predict(test_images)\n",
    "\n",
    "print(classifications[0])\n",
    "print(test_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Bql9fyaNUSFy"
   },
   "source": [
    "#Exercise 6: \n",
    "\n",
    "Consider the impact of training for more or less epochs. Why do you think that would be the case? \n",
    "\n",
    "Try 15 epochs -- you'll probably get a model with a much better loss than the one with 5\n",
    "Try 30 epochs -- you might see the loss value stops decreasing, and sometimes increases. This is a side effect of something called 'overfitting' which you can learn about [somewhere] and it's something you need to keep an eye out for when training neural networks. There's no point in wasting your time training if you aren't improving your loss, right! :)\n",
    "\n",
    "#练习6。\n",
    "\n",
    "请考虑改变训练epochs次数，为有什么影响？\n",
    "\n",
    "试试15个epochs--可能会得到一个比5个epochs更好的模型,损失更小。\n",
    "试试30个epochs--可能会看到损失值停止下降，有时还会增加。这是被称为 \"过拟合 \"的副作用，在训练神经网络时，需要一直注意这个问题。如果损失没有改善，那么浪费时间继续训练是没有意义的，对吧！ :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "uE3esj9BURQe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.2595\n",
      "Epoch 2/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1162\n",
      "Epoch 3/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0798\n",
      "Epoch 4/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0582\n",
      "Epoch 5/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0460\n",
      "Epoch 6/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0353\n",
      "Epoch 7/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0284\n",
      "Epoch 8/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0223\n",
      "Epoch 9/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0201\n",
      "Epoch 10/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0155\n",
      "Epoch 11/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0147\n",
      "Epoch 12/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0117\n",
      "Epoch 13/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0104\n",
      "Epoch 14/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0090\n",
      "Epoch 15/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0086\n",
      "Epoch 16/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0090\n",
      "Epoch 17/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0061\n",
      "Epoch 18/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0068\n",
      "Epoch 19/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0053\n",
      "Epoch 20/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0062\n",
      "Epoch 21/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0054\n",
      "Epoch 22/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0053\n",
      "Epoch 23/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0059\n",
      "Epoch 24/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0047\n",
      "Epoch 25/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0055\n",
      "Epoch 26/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0044\n",
      "Epoch 27/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0043\n",
      "Epoch 28/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0034\n",
      "Epoch 29/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0036\n",
      "Epoch 30/30\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0045\n",
      "313/313 [==============================] - 0s 830us/step - loss: 0.1233\n",
      "[2.1367544e-24 5.0140433e-17 2.2283821e-14 2.6118363e-13 9.7950895e-34\n",
      " 2.3054374e-29 0.0000000e+00 1.0000000e+00 4.8939686e-18 7.3670806e-21]\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(training_images, training_labels) ,  (test_images, test_labels) = mnist.load_data()\n",
    "\n",
    "training_images = training_images/255.0\n",
    "test_images = test_images/255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
    "                                    tf.keras.layers.Dense(128, activation=tf.nn.relu),\n",
    "                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "              loss = 'sparse_categorical_crossentropy')\n",
    "\n",
    "model.fit(training_images, training_labels, epochs=30)\n",
    "\n",
    "model.evaluate(test_images, test_labels)\n",
    "\n",
    "classifications = model.predict(test_images)\n",
    "\n",
    "print(classifications[34])\n",
    "print(test_labels[34])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HS3vVkOgCDGZ"
   },
   "source": [
    "#Exercise 7: \n",
    "\n",
    "Before you trained, you normalized the data, going from values that were 0-255 to values that were 0-1. What would be the impact of removing that? Here's the complete code to give it a try. Why do you think you get different results? \n",
    "\n",
    "#练习7。\n",
    "\n",
    "在训练之前，对数据进行了归一化处理，从0-255的数值变成了0-1的数值。去掉归一化会有什么影响？下面是完整的代码，可以试一试。会得到什么不同的结果？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JDqNAqrpCNg0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.2020\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.0793\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.0528\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.0355\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.0281\n",
      "313/313 [==============================] - 0s 1ms/step - loss: 0.0595\n",
      "[6.4718949e-07 4.8794635e-09 2.1555893e-06 1.3272098e-04 8.9761018e-11\n",
      " 9.7716210e-08 4.4795799e-12 9.9963272e-01 2.2070760e-06 2.2938165e-04]\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "mnist = tf.keras.datasets.mnist\n",
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
    "training_images=training_images/255.0\n",
    "test_images=test_images/255.0\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
    "  tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "])\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "model.evaluate(test_images, test_labels)\n",
    "classifications = model.predict(test_images)\n",
    "print(classifications[0])\n",
    "print(test_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E7W2PT66ZBHQ"
   },
   "source": [
    "#Exercise 8: \n",
    "\n",
    "Earlier when you trained for extra epochs you had an issue where your loss might change. It might have taken a bit of time for you to wait for the training to do that, and you might have thought 'wouldn't it be nice if I could stop the training when I reach a desired value?' -- i.e. 95% accuracy might be enough for you, and if you reach that after 3 epochs, why sit around waiting for it to finish a lot more epochs....So how would you fix that? Like any other program...you have callbacks! Let's see them in action...\n",
    "\n",
    "#练习8。\n",
    "\n",
    "之前在训练模型的时候，你可能会想'如果可以在达到一个期望值的时候停止训练不是很好吗？'--即95%的准确率对你来说可能已经足够了，如果你在3个epochs后达到了这个值，为什么还要坐等它完成更多的训练次数呢....，那么如何解决这个问题？就像其他很多类似程序一样......运用回调函数！让我们来看看它的实际作用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pkaEHHgqZbYv"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.2.0\n",
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.4726\n",
      "Epoch 2/5\n",
      "1864/1875 [============================>.] - ETA: 0s - loss: 0.3590\n",
      "Reached 60% accuracy so cancelling training!\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.3588\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fab29fa4b00>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "class myCallback(tf.keras.callbacks.Callback):\n",
    "  def on_epoch_end(self, epoch, logs={}):\n",
    "    if(logs.get('loss')<0.4):\n",
    "      print(\"\\nReached 60% accuracy so cancelling training!\")\n",
    "      self.model.stop_training = True\n",
    "\n",
    "callbacks = myCallback()\n",
    "mnist = tf.keras.datasets.fashion_mnist\n",
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
    "training_images=training_images/255.0\n",
    "test_images=test_images/255.0\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
    "  tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "])\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')\n",
    "model.fit(training_images, training_labels, epochs=5, callbacks=[callbacks])\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Course 1 - Part 4 - Lesson 2 - Notebook.ipynb",
   "provenance": [
    {
     "file_id": "1yhyaHJIeyutXBuE2HmViuD-tV5EMqzUG",
     "timestamp": 1547798065395
    }
   ],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
